Papers with Recurrent neural network language models
Estimating Marginal Probabilities of n-grams for Recurrent Neural Language Models (D18-1)
Copied to clipboard
| Challenge: | Recurrent neural network language models (RNNs) only estimate probabilities for complete sequences of text, whereas some applications require context-independent phrase probabilities instead. |
| Approach: | They propose a method to alter the RNNLM training to make it more accurate at marginal estimation. |
| Outcome: | The proposed method is effective compared to baselines including the traditional RNNLM probability and importance sampling approach. |
Using Large Corpus N-gram Statistics to Improve Recurrent Neural Language Models (N19-1)
Copied to clipboard
| Challenge: | a technique that uses large corpus n-gram statistics as a regularizer for training a neural network LM on a smaller corpus is effective, and more time-efficient than training on ngrams. |
| Approach: | They propose a technique that uses large corpus n-gram statistics as a regularizer for training on a smaller corpus. |
| Outcome: | The proposed technique is effective and more time-efficient than training on a larger corpus. |